The inventors of the present invention have determined that multiview camera calibration has been an engineering challenge for a very long time. There are many tools and techniques that are available for calibration of multiview, especially stereo-based, camera systems. Calibration algorithms for stereo rigs traditionally aim at capturing the intrinsic and extrinsic parameters that are associated with a given stereo rig. Intrinsic parameters are parameters that are associated with the individual cameras themselves (focal length, pixel pitch, camera resolution, etc.) The extrinsic parameters as illustrated in FIG. 1 are ones that define the relationships between the various camera nodes in a stereo rig. These parameters include rotation parameters, like pitch, yaw, and roll. They also include translation parameters, like distance in x, y and z, of the nodes relative to each other. There are also many more parameters that can be estimated if they are needed in a three-dimensional scene analysis. Calibration aims at estimating such parameters based on a series of observations. Most calibration approaches require known three-dimensional information from the scene. Most of the time, the process calibration results in a relatively accurate system, capable of three-dimensional depth reconstruction.
The inventors of the present invention have further recognized that using these conventional approaches, stereo calibration has not been perfected in any way, and likely, never will. This is because stereo calibration intrinsically assumes parameters as part of its optimization process for triangulation. However, some of these parameters can not be estimated. At best, a tolerance stackup during manufacturing may be used for such a process. A tolerance stackup would include all of the tolerances that are associated with a manufacturing process, including all the hardware specs and components. For highly accurate three-dimensional reconstruction requirements, such parameters are inherently incorrect because some of the tolerances are too small a value for stereo, Time-of-flight or even structured light distance based computations. Take, for instance, the example of the extrinsic yaw parameter. Yaw, is a rotational angle measure governing horizontal rotation between left and right views in a stereo camera. Yaw estimation, along with other intrinsic and extrinsic parameters, is making assumptions about the system that are otherwise untrue. Key amongst such assumptions, for instance, is one in which the optical axis is aligned normally to the image sensor plane. The angle between the optical axes represents the yaw value. Reality, however, makes little sense or use of such cases. In fact, there is an inherent tilt between the optical axis and the image sensor plane that is not captured by the yaw estimation, in most any work that is associated with calibration. Any such remaining error in calibration will be referred to as residual error. The inventors of the present invention have determined that mitigating this residual error would result in improved accuracy of the solution.
It would therefore be beneficial to present a method and apparatus for overcoming the drawbacks of the prior art.